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"IMAGE VIEWING SYSTEM AND METHOD FOR GENERATING FILTERS FOR FILTERING IMAGE 
FEATURES ACCORDING TO THEIR TYPE" 

Description 

Field of the Invention 

The invention relates to an image processing system and an image processing method to 
generate adaptive filters, whose supports are adapted to the type of features of interest in an image 
to be processed. The invention more particularly relates to such processing system and method that 
are applied to medical images. The invention also relates to a medical examination apparatus having 
such an image processing system and means for display. The invention finds an application in the 
Industry of image processing systems and medical examination apparatuses. 

Background of the Invention 

A method to smooth and filter Images Is disclosed in the publication "A Framework for 
Anisotropic Adaptive Filtering and Analysis of Sequences and volume" In IEEE International 
Conference on Acoustics, Speech and Signal Processing (ICASP); San Francisco, March 23-26, 1992; 
p.469-472, vol.3, by H. Knutsson, L. Haglund and G.H. Granlund. The approach described in the cited 
publication makes use of quadrature and steerable filters having a set of different orientations 
including at least three different orientations in the case of 2-D image processing, and at least six 
different orientations In the case of 3-D image processing. The algorithm written from this approach 
requires the application of all the filters involved In the filtering of one Image. The resulting algorithm 
complexity has been found to be high. So, the calculation time Is particularly long. 

Summary of the Invention 

The present Invention has for an object to provide an Image viewing system having 
processing means to generate an adaptive filter for filtering an image comprising features of different 
type. By type of the image features, it is to be understood either anisotropic features such as features 
having elongated oriented shapes or Isotropic features such as features rather showing circular or 
spherical shapes. In particular, it is an object of the invention to provide an image viewing system 
having analysis means for analyzing the content of a noisy image representing various objects and for 
determining the type of the features of interest of the objects in the image; and having synthesis 
means for synthesizing filters whose supports are adapted to processing the different features 
according to their different types, forfilterlng noise or enhancing these features In the image. 

It is an object of the Invention to provide analyzing means to estimate at each image point a 
probability measure of the presence of a feature of interest; then, from said probability measure, to 
determine a weighting control model for the user to control adaptive kernels at each image point. 
Such an image viewing system is claimed in claim i. 
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It is a further object of the Invention to yield synthesizing means to provide the user with 
filtering means, among which: 

first filtering means called "pre-mixing filtering means" comprising combining means 
dependent on the type of the image features having inputs for the weighting control means and the 
image data and having an output for weighted adaptive kernels adapted to the type of the image 
features; and/or 

second filtering means called "post-mixing filtering means" comprising both isotropic and 
anisotropic filtering means applied independently of the type of the image features, whose outputs are 
combined at each image point and adapted using the weighting control means. 

A characteristic of the system of the invention is that at each point, the oriented filter means 
and the isotropic filter means are mixed according to the degree of feature type present at. each point, 
such as for example the degree of anisotropic type and isotropic type of said point Several problems 
are solved during those two phases of analysis and synthesis: 

The analysis is simplified however taking into account different kinds of information such as 
feature orientation and feature dimensions. The control means provided to the user allows him to fully 
specify the type of the features that he wishes to enhance. It allows him to fine-tune the filter in an 
extremely fast way, he. much faster than the actual filtering time. 

The solution to these problems Is based on: . . - „ 

The use of the control model based on the estimation of a vector (Fi,..F m ) T of the pixel-level 
probabilities of the presence in the image of a number m of features of interest. This feature 
probability vector Is captured by a set of m operators f lt f m . The design of those operators depends 
on the type of the features to be captured. 

The control model Is completed by an explicit user-driven combination means X u of the 
feature probabilities into a k-dimension control vector (Ci,...,c k ) T . This combination specifies the user's 
wishes in terms of: "which feature or feature combination should be enhanced and with what 
strength". 

Selector means may be used to select filtering means chosen among two chains of proposed 
filtering means. The first filtering chain, called "pre-mixing filtering means", starts from the real-time 
image and produces an output signal R H using a real-time adaptive-enhancing filter kernel H. The 
definition of this kernel H is specified by the above defined combined control vector (Ci,...,c k ) T using a 
combination operator X H . The second filtering chain, called "post-mixing filtering means", comprises a 
combining operator Xg, which combines the outputs of m predefined adaptive-enhancing filters g, and 
produces an output signal Rg. Each of these m predefined adaptive-enhancing filters g, is dedicated to 
the enhancement of a specific feature. Their combination is also driven by the control vector (c!,...,c k ) T 
via the combination operator Xg. This chain allows extremely fast tuning of the filter by the user since 
the adaptive filters gi,..,,gh are computed Independently of the user tuning. Only the combination 
operator Xg needs to be recomputed at every new control adjustment 

The system of the invention has very powerful means to constitute adaptive filters for 
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filtering features of different types such as anisotropic oriented features or isotropic areas. These 
filtering means show very low complexity of calculation, resulting In substantial computation saving. In 
addition, me synthesis means of the system of the invention gives to the user the choice between first 
filtering means and second filtering means whose complexities are different and whose computation 
loads are also different. Besides, the computations means according to the invention can be efficiently 
implemented using very simple techniques. Another advantage of the system of the invention Is that 
the required computation time of the second filtering means is especially short. This system of the 
invention can constitute a very fast and general multidimensional adaptive filter. 

The second embodiment of the system of the invention can be implemented so that it Is 
minimally demanding in terms of inputfoutput, thus making a very good filter candidate for general 
purpose processor implementation. In fact, the system of the invention presents the advantage that it 
does not require any memory means. Since at the present time, processing means are very efficient 
for producing results of calculations, but are not very efficient for extracting data from memories, 
hence, the system of the Invention, which only comprises calculation means, is very efficient. 

In these advantageous embodiments of the invention, the user disposes of control means 
for interacting with the analysis means to select the weight to be applied to the filtering of the 
features of interest in the image and/or to the synthesis means. 

The invention may refer to image data in which each data point (pixel or voxel) Is attributed 
a single scalar intensity value. This can be extended to multi-spectral data in which several values are. 
attributed to each data point For example, in colored images, one may refer to Red, Green and Blue 
components. Multidimensional data can be dealt with, such as one dimension, spatial 2-D, spatial 3-D, 
spatial multidimensiona. data plus intensity dimensional data and/or plus time dimensional data. When 
one of the dimension is time and when one aims at real-time results, the method of the Invention may 
have recursive steps for using data recorded at previous instants. 

Brief Description of the Figures 

The invention is described hereafter in detail in reference to diagrammatic figures, wherein : 
RG.l is a functional block diagram Illustrating the means of the image viewing system; 
FIG.2 illustrates the steps carried out by the viewing system of the invention; 
FIG.3 is a functional block diagram of a viewing system and a medical apparatus according 



to the Invention. 



Description of Embodiments 



The invention relates to an image viewing system having processing means to generate 
filters whose supports are adapted according to local type of features of interest in an image to be 
processed. The invention relates to such a system for adaptive filtering of multidimensional signals for 
quality improvement application including noise reduction and the enhancement of valuable details. 
This system has processing means for achieving this goal In two steps: first means for signal analys.s 
and using the results of this analysis, second means for filter synthesis. In particular, the invenhon 
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relates to an Image viewing system having analysis means for analyzing the content of a noisy image 
representing various objects, in order to determine the type of features of interest in an image; and 
having synthesis means for synthesizing filters with adapted supports apt to filter noise or enhance 
the features of different type in the image. The analysis means analyses the image signal in order to 
determine local properties of the image to be filtered. The synthesis means realizes a filtering kernel, 
which varies at for each current point of the image and is adapted to the local properties of the image 
signal, while scanning the different points of the image during the filtering process. 

The system comprises means of acquisition of image data including point coordinates, 
denoted by x, and an intensity value, denoted by I(x), for each image point x. The proposed system 
deals with a multi-dimensional image, whose dimension is denoted by d. It is not excluded that d - 2. 
For a d-dimensional image, each data point is defined by a d-tuple point x associated to coordinates 
of the form (x lf x^.-Xd) where each of the components x lt X2,...x<, varies In a predefined range, for 
instance 0< Xl <I, ; 0<x 2 <li ; 0 <x d < I* To each such d-tuple, corresponds an Image value, which 
may be a single scalar value. In the case of a monochromatic image, the scalar value is the luminance 
intensity. For a color image, the image value may be represented by a set of three intensities 
representing the Red, Green and Blue components. 

FIG.l is a block diagram of the image processing system of the Invention. Referring to 
FIG:l, the input image data I(x) are processed using the following processing means: 
1) Analysis means for analyzing the content of the original image. 
The Analysis means comprises a bank of a number m of operators, denoted by fi,..,f,,..,f m 
and referenced 5 in FIG.l. The bank 5 estimates at each current point of the Image a probability of 
presence of features of Interest. As feature of interest, one is to understand either an anisotropic 
feature which is an elongated oriented feature, such as for example in a displayed medical image 
representing vessels, an edge or a border of a vessel; or an isotropic feature which may be rather a 
circular or spherical feature, such as an aneurysm or a vessel perpendicular to the plane of the 
displayed image. 

More generally, for a 2-D image signal, anisotropic features might include crest, valleys, 
crest Intersections, pass, plateau borders. It is to be noted that these typologies (crest, valleys, etc..) 
refer to 2-D Image signals where the intensity of the 2-D image is represented In a third dimension. 
Hence, the intensity is figured out as the height dimension in a landscape. Isotropic features might be 
"flat" isotropic areas, where flat means having constant intensity in the height dimension. For a 3-D 
image signal, the features mighteven be more diverse, Including surfaces oriented in any directions, 
and 3-D planar or non-planar (skew) curves. 

The outputs of the bank of operators 5 are formed by a number m of probability values, 
denoted by F 1 ,..,F I ,..,F m , for the m types of different features to be present in the image to be filtered. 

In an example, In an image, two types of features may be present, such as edges and 
isotropic areas. The probability of presence of edges is expressed by ot, and the probability of 
presence of isotropic areas is expressed by (1 - a). 
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in another example, In an image, three types of features may be present, such as edges, 
edge intersections, and isotropic areas. The probability of presence of simple edges is expressed by a 
(alpha), and the probability of presence of isotropic areas is expressed by P (beta), and the probability 
of presence of edge intersections is expressed by y (gamma). The total probability value is preferably 

normalized so that: 

a + Y - 1 (1) 

The generalfonlL giving the probability of presence of different features may be expressed by: 

l=m (2) 

The type of the operators of the bank 5 depends on the type of the features of Interest. It is 
not the object of the Invention to specify these types very strictly. In the above described case when 
edges and isotropic areas may be found as features In the image, the probability a of presence of 
edges can be determined for example by calculating the intensity gradient at each point in the image. 

Hence, according to the invention, a number m of features of different type are found In the 
image using a corresponding number m of operators 5 that depends on the type of the features. 

A combining operator 10, called user operator, denoted by X u , further receives at its Input 
the m probabilities F, of presence of features of different type and provides at its outputs a control 
vector of k components Q,..,Q,.,C,, This control vector 11 of k components, denoted by V c , allows to 
controlling the adaptive kernels of the niter to be constructed according to the invention. The number 
k of vector components is such that m * k, depending on whether the user wants to group the 
filtering of different types of features together or not Using this control vector V a the user w.li be 
able to define the feature or the group of features that he wants to filter or to enhance and the 
strength of the filtering or enhancement. For instance, in the above-described example of an .mage 
comprising edges and circular features, the user may choose to enhance the isotropic features with 
strength of 0.8 and the anisotropic features with strength of 0.2. 

Hence, the analyzing means provides the control vector V & which is apt to control the 
adaptive kernel to be passed on each point in the image. It Is to be noted that this control vector 
depends on the type of the features and not on the direction of particular features such as the 

direction of anisotropic features. 

2) Synthesis means for synthesizing oriented filters. 

The synthesis means comprises two possible chains of means for generating an adaptive 
filter The first chain is denoted by H and called "pre-mixing filtering chain of generating adaptive 
filter" The second chain is denoted by G and called "post-mixing filtering chain of generating adaptive 
filer". The user has means 158 called user control interface, for selecting the first or the second chain. 

2.1) Pre-mixing filtering chain of generating adaptive filter. 
This chain comprises a combining operator denoted by X„, referred to as 30, which receives 
at its input the control vector V c and the image data I(x) and which provides at its output an 
adaptive kernel. This adaptive kernel is adapted to the orientation of the anisotropic oriented features 
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and/or to the dimensions of the isotropic features to be filtered or enhanced. In addition, this adaptive 
kernel is adapted to respect the weighting parameters defined by the control vector V C/ since V c Is 
used on top of orientations and/or dimensions of the features. 

Those skilled in the art know means to synthesize a filter defined by a probability such as a 
and by structure tensors. A map of the a probabilities defines the moments of a target tensor. From 
the target tensor, a filter is synthesized where the tensor eigen vectors give the local orientation and 
where the tensor eigen values give indications of the strength in the different orientations. Hence, the 
information of strength along different directions that are given by the control vector may be used to 
control a tensor. According to the invention, a tensor is synthesized, in order to detect anisotropic or 
isotropic features, so that it is aligned on a local tangential direction. 

In an example, the number m of features of the image is m = 2, the first feature correspond 
to F„ and the second feature to F 2 . The tensor is defined by a tangential moment m and by a 
perpendicular moment pp. The minimum and maximum moment are respectively p MIM and p„ax. The 
moment are given by the following formulae: 

Pt = Pmin + a (nt max - Pmin ) (3a) 
Mp ~ Pmin + a ((ip max - Umin )• (35) 
When a = O, nt = ^ = p MWI ^ 

hencethe tensor is isotropic since the strength, is the same in the directions t and. p. 

When a = 1, p* = p, max, (5a) and 

Pp = Hp max (5b) 
hence the tensor is completely oriented. 

With the above-defined tensor, the filter can be realized: this filter will clearly depend on the 
type of the features that are present in the image and that the user wants to favor. In this chain of 
filter generation, the information about the type of features is mixed to perform the generation of the 
filter kernel H referred to as 35. The result 3 of the filtering is given by H(x). 

2.2) Post-mixing filtering chain of generating adaptive filter. 

In this second chain of filter generation, instead, the image data are first filtered using a 
bank 15 of a number m of different filtering means denoted by g 1 ,..,g i ,.. / g m . The number of these 
filtering means corresponds to the number m of features of different type to be processed. The 
outputs, denoted by G u ..,G lt ..,G m , of the different filtering means are mixed. Each different filtering 
operation relates to a given Identified feature. In this chain, adaptive or non-adaptive filters are used. 
Each of these filters Is specialized in the filtering of one given type of feature, that has been previously 
Identified, such as edges, intersections and circular or spherical features. 

The combination of the outputs of the filters of the bank 15 is performed using a 
combination operator Xg. This combination operator Xg, referred to as 20, also receives the control 
vector V c of the k components C» to C k . 

For instance, the combination may be a simple weighted sum of the results of the filter bank 
15. This combination operator Xg is very importantelement of this chain, because it allows to 
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changing very easily the weights related to the different type of features in the final result 2 denoted 
by G(x), whichever are the filtering means of the bank of filters 15. The filtering means of the bank 15 
are applied independently of the types of the features examined in the image. The filtering operations 
using the bank 15 are realized in a preliminary step and the weight variations are performed while 
combining the results G 1/ ..,Gi,..,G nw by using the control vector V c . 

In an example, an image comprising anisotropic features and isotropic features can be 
filtered using two types of filters denoted by gi and g 2 in the bank of filters 15. G 1 is for example the 
output of the oriented filter g x and G 2 is the output of the isotropic filter g 2 . The filter g z is not 
necessarily adaptive, it may be a simple Gaussian kernel. The user may choose a filtering strength for 
the anisotropic features given by a. In this case, the result at the output of the combining operator 
Xg, referred to as 20, Is given by the formula: 

G(x) = a(x) Gi(x) + (1 - a) G 2 (x) (6) 

Since the value ce(x) depends on the choice of the user, the user can change the result G(x) 
according to its desire to improve the filtering strength or to improve the enhancement of the different 
types of features. This operation of varying the filtering strength can be achieved without performing 
any new calculation of the results Gi,..,Gi,..,GU of the different filters gi,..,gi,..,g m of the bank of filters 
15. 

In an example of realization, the oriented filter gi can be made using structure tensors with: 
lit = a constant (7a), and 

Hp = another constant (7b) 
It is to be noted that the isotropic filter g 2 does not depend on pt or jip values. 
3) Image processing method. 

Referring to FIG.2, the image processing method, for generating a multidimensional 
adaptive oriented filter to process image data in a number d of dimensions, operated by the above- 
described system comprises steps of: 

analyzing (5, fi) the image to estimate at each image point a probability measure (F|) of the 
presence of a type, of feature of interest and to determine from said probability measure a weighting 
control model (10) yielding a weighting control vector (11, V c ) for the user to control synthesized 
adaptive kernels at each image point; and 

synthesizing filter kernels at each image point adapted to the type of the features of 
interest, which filter kernels are controlled by the weighting control vector. 

The user may select a chain of steps for generating the filter kernels called "pre-mixing 
filtering chain of steps", comprising a combination operation (30, X H ) dependent on the type of the 
image features for combining the weighting control vector (11, V c ) and the image data P(x)] and 
yielding weighted adaptive kernels (35, H) adapted to the type of the image features to produce the 
filtered image signal [H(x)]. 

The user may select another chain of steps for generating the filter kernels called "post- 
mixing filtering chain of steps", comprising performing different. types of filtering operations [15, gi)], 
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such as isotropic and anisotropic filtering operations, applied independently of the type of the image 
features. These filtering operations yield signals (G { ) that are further combined, at each image point of 
the image, using the weighting control vector (11, V c ), in order to produce the filtered image signal 
[G(x)] adapted to the types of features. 

4) Medical viewing system and apparatus 

The above-described means and steps can be implemented in the viewing system of the 
invention. Fig.3 shows the basic components of an embodiment of an image viewing system in 
accordance to the present invention, incorporated in a medical examination apparatus. The medical 
examination apparatus 151 may inciude a bed 110 on which the patient lies or another element for 
localizing the patient relative to the imaging apparatus. The medical imaging apparatus 151 may be a 
CT scanner or other medical imaging apparatus such as x-rays or ultrasound apparatus. The image 
data produced by the apparatus 151 is fed to data processing means 153, such as a general-purpose 
computer, that carries out the steps of the method. The data processing means 153 is typically 
associated with a visualization device, such as a monitor 154, and an input device 155, such as a 
keyboard, or a mouse 156, pointing device, etc. operative by the user so that he can interact with the 
system. The data processing device 153 is programmed to implement a method of processing medical 
image data according to invention. In particular, the data processing device 153 has computing means 
and memory means to perform the steps of the method. A computer program product having pre- 
programmed Instructions to carry out the method may also be implemented. 

The drawings and their description herein before illustrate rather than limit the Invention. It 
will be evident that there are numerous alternatives that fall within the scope of the appended claims. 
Moreover, although the present invention has been described in terms of generating image data for 
display, the present invention is intended to cover substantially any form of visualization of the image 
data including, but not limited to, display on a display device, and printing. Any reference sign in a 
claim should not be construed as limiting the claim. 
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Claims 

1. Image processing system for generating a multidimensional adaptive oriented filter to be 
applied to the point intensities of an image formed In a number d of dimensions, comprising: 

analyzing means comprising means (5, f,) to estimate at each image point a probability 
measure (F,) of the presence of a type of feature of interest and to determine from said probability 
measure a weighting control model (10) yielding a. weighting control vector (11, V c ) for the user to 
control synthesized adaptive kernels at each image point; and 

synthesizing means for generating filter kernels at each Image point adapted to the type of 
the features of interest, which filter kernels are controlled by the weighting control vector. 



2. The image processing system of Claim 1, wherein the synthesizing means comprises means 
for generating: 

filtering means called "pre-mixing filtering means" comprising combining means (30, X H ) 
dependent on the type of the image features having inputs for the weighting control vector (11, V c ) 
and the image data U(x)3 and having an output for weighted adaptive kernels (35, H) adapted to the 
type of the image features to produce the filtered image signal [H(x)]. 

3. The image processing system of one of Claims 1 or 2, wherein the synthesizing means 

comprises means for generating: 

filtering means called "post-mixing filtering means" comprising both isotropic and anisotropic 
. filtering means [15, g,)3 applied independently of the type of the image features, whose outputs (G,) 
are combined at each image point and adapted using the weighting control vector (11, V c ) to produce 
the filtered image signal [G(x)]. 

4. The image processing system of one of Claims 1 to 3, wherein the analyzing means 
comprises a number m of operators (fi,..,fi,..,f m ), which outputs at each current point of the Image a 
probability value OWWU of presence of features of interest among the m types of different, 
features in the image to be filtered. 

5. The image processing system of one of Claims 1 to 4, wherein the analyzing means 
comprises a combining operator (10, XJ, called user operator, which receives at its input the m 
probabilities (Fi,..,Fv.,F m ) of presence of different types of features for forming the weighting control 
model, which provides at its outputs the control vector (Vc) of k components (Ci,..,Q,..,C0, for 
controlling the adaptive kernels of the synthesized filters. 



6. The image processing system of one of Claims 1 to 5, wherein the features of interest are 
chosen among anisotropic features and isotropic features. 
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7. The (mage processing system of Claim 6, wherein the synthesizing means comprises, in the 
"pre-mixing filtering means", a combining operator (30, X H ), which receives at its Input, the control 
vector V c and the image data I(x) and which provides at its output an adaptive kernel (H) that is 
adapted to the orientation of the anisotropic oriented features and/or to the dimensions of the 
isotropic features to be filtered or enhanced by the weighting parameters defined by the control 
vector (V c ). 



8. The image processing system of Claim 6, wherein the synthesizing means comprises, In the 
"post-mixing filtering means", a number m of different filtering means (gi,..,g„..,g m ), which are 
applied independently of the types of the features examined in the image, whose number m 
corresponds to the number m of features of different types to be processed and whose outputs. 
(Gi,..,G,,..,G ro ) are mixed in a combination operator (Xg), which is controlled by the control vector (V c ) 
to produce the filtered image signal [G(x)]. 

9. The image processing system of Claim 8, wherein the combination operator (Xg) is a 
weighted sum of the results of the different filtering means (gi,..,gi,..,g m ). 

10. The Image processing system of one of Claims 2 to 9, wherein a user control interface (158) 
is provided for the user to control the weighting parameters (C^C-.A) of the control vector (V c ); 
for selecting the "pre-mixing filtering means" and/or the "post-mixing filtering means"; for varying the 
strength of filtering or enhancement related to the different type of features at the output of the 
combination operator (Xg) while combining the results (Gi,..,G„..,G m ) to produce the filtered image 
signal [G(x)] In the "post-mixing filtering means"; and/or at the input of the combination operator (X H ) 
in the "pre-mixing filtering means". 

11. Image processing method for generating a multidimensional adaptive oriented filter to 
process image data, using a system as Claimed in one of Claims 1 to 10. 

12. Medical examination apparatus comprising means to acquire d-dimensional image data 
P(x)] 7 a system as Claimed in one of Claims 1 to 10 and further comprising a display system (154) for 
visualizing processed images and user control means (158) for selecting weighting parameters and/ or 
acting on the user operator (10) and/or the selection unit (40). 

13. A computer program product comprising a set of instructions for carrying out the method as 
claimed in Claim 11. 
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Abstract 

Image processing system for generating a multidimensional adaptive oriented filter to be 
applied to the point intensities of a d-dimensional image, comprising analyzing means with means (5, 
f ( ) to estimate at each image point a probability measure (Fi) of the presence of a type of feature of 
interest and a weighting control model (10) issuing a weighting control vector (ll f V c ) constructed 
from said probability measure, for the user to control synthesized adaptive kernels at each image 
point; and synthesizing means for generating the filter kernels at each image point adapted to the 
type of the features of interest, whose filtering strength is controlled by the weighting control vector. 
The system may comprise a selection unit (40) for the user to select synthesizing means for 
generating "pre-mixing filtering means" comprising combining means (30, X H ) dependent on the type 
of the image features having inputs for the weighting control vector (11, V c ) and the Image data [I(x)] 
and having an output for weighted adaptive kernels (35, H) adapted to the type of the image features 
to produce the filtered Image signal [H(x)]; and/ or "post-mixing filtering means" comprising both 
isotropic and anisotropic filtering means [15, gs)] applied independently of the type of the image 
features, whose outputs (Gj) are combined at each image point and adapted using the weighting 
control vector (11, V c ) to produce the filtered image signal [G(x)]. 
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